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Assistant Commissioner for Patents 
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Sir: 

This is a Request for filing a Continuation application under 37 CFR 1.53(b) of pending prior application 
Serial No. 09/420.262 , filed on October 18. 1999 , entitled CLASSIFICATION OF INFORMATION AND USE OF 
CLASSIFICATIONS IN SEARCHING AND RETRIEVAL OF INFORMATION , by the following named 
inventor(s): .lEREMY D.F. HTNMAN: KATHERINE E. HINMAN: ROLF W. KAISER: CLYDE SUTLIFF; 
GEOFFREY R. STANFIELD: THOMAS S. EVERIST. Ill: HJDITH D. MAHTABAN: PAUL DEEDS: FRANK 
FLANNERY: KAREN A. LINDENBERG: JAYMEE MURISET: PAUL ED MONDSON: ANDREA BEYER; 
MELISSA THKLEMAOUE: ERIC BASSMAN: and DAVID RIVELLL Note : Inventorship is different in this 
continuation. 

1 . □ I hereby state that the enclosed copy of this prior application is a true copy of die above-identified prior 

application. 

2. Oath or Declaration 

a. Newly executed (original or copy) 

b. Q Copy from a prior application (37 CFR 1 .63(d)) 
i. Q Deletion of inventory s> 

Signed statement attached deleting inventor(s) named in the prior application, see 37 
CFR 1.63(d)(2) and 1.33(b). 

3. 13 Incorporation By Reference (useable ifBox 2b is checked) 

The entire disclosure of the prior application, from which a copy of the oath or declaration is suppHed 
under Box 2b, is considered as being part of the disclosure of the accompanying application and is hereby 
incorporated by reference therein. 

4. 13 Assignment is enclosed. 

5- □ An Information Disclosure Statement and PT01449 Form are submitted herewith. 
6. Q Cancel claims . 
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Independent Claims 
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Basic Application Fee 
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Total Application Fee 
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TOTAL APPLICATION FEE DUE 
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7a. IXI Enclosed is a Verified Statement to establish small entity status under 37 CFR 1 .9 and 37 CFR 1 .27, 

7b. □ A verified Statement to establish small entity status under 37 CFR 1 .9 and 37 CFR 1 .27 was filed in 
prior application and such status is still proper and desired. 

8. ^ The Commissioner is hereby authorized to charge fees under 37 CFR 1.16 and 1.17 which may be 

required, racluding any extension of time fees to maintara the pendency of the parent application Serial 
No. 09/420,262 or credit any overpayment to Deposit Account No. 50-0385 . 

9. ^ Amend the specification by inserting before the first line the sentence: 

-This application is a contmuation of Application Serial No. 09/420,262 filed October 18, 1999.-- 

10. □ Priority of Application Serial No. filed on , in is claimed under 35 USC 119. The certified priority 

document(s) were filed in Serial No. on , 

1 1 . □ The prior application is assigned of record to 
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14- □ ^ petition, fee and response has been filed to extend the term in the pending prior appUcation until . 

Address all future communications to: (May only be completed by applicant, or attorney or agent of record) 

McDermott, Will & Emery 
600 13"^ Street, N.W. 
Washington, DC 20005-3096 



600 13'^ Street, N.W. 
Washington, DC 20005-3096 
(408)271-2300 CJP:ka 
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Facsimile: (408)271-2310 



Respectfully submitted, 
MCDERMOTT, WILL & EMERY 
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Christopher J. Palermo 
Registration No. 42,056 
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Kim Area _„ .„ 
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For: CLASSIFICATION OF INFORMATION AND USE OF CLASSIFICATIONS IN SEACHING AND 

RETRIEVAL OF INFORMATION 

VERIFIED STATEMENT (DECLARATION) CLAIMING SMALL ENTITY STATUS 
(37) CFR 1.9(f) and L27(c) - SMALL BUSINESS CONCERN 

I hereby declare that I am 

[ ] the owner of the small business concern identified below: 

[X] an official of the small business concern empowered to act on behalf of the concern 
identified below: 

NAME OF ORGANIZATION: MONGOMTTSTC. INC. 

ADDRESS OF ORGANIZATION: Q4Q Hamilton Street. Menio Park. CA 9402 5 

I hereby declare that the above identified small business concern quaUfied as a small business concern as defined in 
l^FR 121.3-18, and reproduced in 37 CFR 1.9(d), for purposes of paying reduced fees under section 41(a) and (b) 
oSritle 35, United States Code, in that the number of employees of the concern, including those of its affiliates, does 
n<i exceed 500 persons. For purposes of this statement, (1) the number of employees of the business concern is the 
average over the previous fiscal year of the concern of the persons employed on a full-time, part-time or temporary 
b^is during each of the pay periods of the fiscal year, and (2) concerns are affiliates of each other when either, 
dfrpctly or indirectly, one concern controls or has the power to control the other, oral third party or parties controls or 
h^ the power to control both. 

Tjtereby declare that rights under contract or law have been conveyed to and remain with the small business concern 
iintified above with regard to the invention, entitled CLASSTFTCATION OF INFORMATION AND USE OF 
d!l ASSIFICATTONS IN SEACHING AND RETRTRVAI OF INFORMATION bv inventor(s) JEREMY D.F. 
hInMAN: ROLF W. KAISFWt GEOFFREY R. STANFTFLD: PAUL DEEDS; FRANK FL ANNERY; and 
1 ^ VTD REVFLLI described in 

[X] the specification filed herewith; 

[ ] application Serial No. , filed , 

[ ] patent no. , issued . 



If the rights held by the above identified small business concern are not exclusive, each individual, concern or 
organization having rights to the invention is listed below* and no rights to the invention are held by any person, other 
than the inventor, who could not quahfy as an independent inventor under 37 CFR 1.9(c) if that person made the 
invention, or by any concern which would not qualify as a small business concern under 37 CFR 1.9(d) or a nonprofit 
organization under 37 CFR 1.9(e). *NOTE: Separate verified statements are required fi:om each named person, 
concern or organization having rights to the invention averring to their status as small entities. (37 CFR 1.27) 
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willful false statements and the like so made are punishable by fine or imprisonment, or both, under section 1001 of 
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CLASSIFICATION OF INFORMATION AND USE OF CLASSIFICATIONS IN 
SEARCHING AND RETRIEVAL OF INFORMATION 



FIELD OF THE INVENTION 
5 The present invention relates to classification of information and the use of 

classifications in searching and retrieval of information. The invention relates more 
specifically to classification of musical compositions and retrieving information about one or 
more musical compositions that are qualitatively similar to a first musical composition. 

10 BACKGROUND OF THE INVENTION 

Classifying information that has subjectively perceived attributes or characteristics is 
difficult. When the information is one or more musical compositions, classification is 
compUcated by the widely varying subjective perceptions of the musical compositions by 
different listeners. One listener may perceive a particular musical composition as "haimtingly 

1 5 beautiful" whereas another may perceive the same composition as " annoyingly twangy." 

In the classical music context, musicologists have developed names for various 
attributes of musical compositions. Terms such as adagio, fortissimo, or allegro broadly 
describe the strength with which instruments in an orchestra should be played to properly 
render a musical composition from sheet music. In the popular music context, there is less 

20 agreement upon proper terminology. Composers indicate how to render their musical 
compositions with annotations such as brightly, softly, etc., but there is no consistent, 
concise, agreed-upon system for such annotations. 

As a result of rapid movement of musical recordings from sheet music to pre- 
recorded analog media to digital storage and retrieval technologies, this problem has become 

25 acute. In particular, as large libraries of digital musical recordings have become available 
through global computer networks, a need has developed to classify individual musical 
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compositions in a quantitative manner based on highly subjective features, in order to 
facilitate rapid search and retrieval of large collections of compositions. 

Musical compositions and other information are now widely available for sampling 
and purchase over global computer networks through online merchants such as Amazon.com, 

5 Inc., bamesandnoblexom, cdnow.com, etc. A prospective consumer can use a computer 
system equipped with a standard Web browser to contact an online merchant, browse an 
online catalog of pre-recorded music, select a song or collection of songs ("album"), and 
purchase the song or album for shipment direct to the consumer. In this context, online 
merchants and others desire to assist the consumer in making a purchase selection and desire 

10 to suggest possible selections for purchase. However, current classification systems and 
search and retrieval systems are inadequate for these tasks. 

A variety of inadequate classification and search approaches are now used. In one 
approach, a consumer selects a musical composition for listening or for purchase based on 
past positive experience with the same artist or with similar music. This approach has a 

1 5 significant disadvantage in that it involves guessing because the consumer has no familiarity 
with the musical composition that is selected. 

In another approach, a merchant classifies musical compositions into broad categories 
or genres. The disadvantage of this approach is that typically the genres are too broad. For 
example, a wide variety of qualitatively different albums and songs may be classified in the 

20 genre of " Popular Music" or " Rock and Roll" 

In still another approach, an online merchant presents a search page to a client 
associated with the consumer. The merchant receives selection criteria from the client for use 
in searching the merchant's catalog or database of available music. Normally the selection 
criteria are hmited to song name, album title, or artist name. The merchant searches the 

25 database based on the selection criteria and returns a Ust of matching results to the cUent. The 
chent selects one item in the list and receives further, detailed information about that item. 
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The merchant also creates and returns one or more critics' reviews, customer reviews, or past 
purchase information associated with the item. 

For example, the merchant may present a review by a music critic of a magazine that 
critiques the album selected by the client. The merchant may also present informal reviews of 

5 the album that have been previously entered into the system by other consumers. Further, the 
merchant may present suggestions of related music based on prior purchases of others. For 
example, in the approach of Amazon.com, when a client requests detailed information about 
a particular album or song, the system displays information stating, "People who bought this 
album also bought . . followed by a list of other albimis or songs. The list of other albums 

10 or songs is derived from actual purchase experience of the system. This is called 
"collaborative filtering." 

However, this approach has a significant disadvantage, namely that the suggested 
albums or songs are based on extrinsic similarity as indicated by purchase decisions of 
others, rather than based upon objective similarity of intrinsic attributes of a requested album 

15 or song and the suggested albums or songs. A decision by another consumer to purchase two 
albums at the same time does not indicate that the^wo albums are objectively similar or even 
that the consumer liked both. For example, the consumer might have bought one for the 
consumer and the second for a third party having greatly differing subjective taste than the 
consumer. As a result, some pundits have termed the prior approach as the greater fools" 

20 approach because it relies on the judgment of others. 

Another disadvantage of collaborative filtering is that output data is normally 
available only for complete albums and not for individual songs. Thus, a first album that the 
consumer likes may be broadly similar to second albimi, but the second album may contain 
individual songs that are strikingly dissimilar firom the first album, and the consumer has no 

25 way to detect or act on such dissimilarity. 
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Still another disadvantage of collaborative filtering is that it requires a large mass of 
historical data in order to provide useful search results. The search results indicating what 
others bought are only useful after a large number of transactions, so that meaningful patterns 
and meaningful similarity emerge. Moreover, early transactions tend to over-influence later 
5 buyers, and popular titles tend to self-perpetuate. 

In a related approach, the merchant may present information describing a song or an 
album that is prepared and distributed by the recording artist, a record label, or other entities 
that are commercially associated with the recording. A disadvantage of this information is 
that it may be biased, it may deliberately mischaracterize the recording in the hope of 
1 0 increasing its sales, and it is normally based on inconsistent terms and meanings. 

Accordingly, there is a need for an improved method of classifying information that is 
characterized by subjective criteria, such as musical compositions. There is a related need for 
an improved schema or set of attributes that may be used in such classification. 

There is a need for a mechanism that can enable a cHent to retrieve information about 
15 one or more musical compositions and that avoids the disadvantages and drawbacks of the 
approaches described above. 

There is a particular need for a mechanism that can enable a client to retrieve 
information about one or more musical compositions that are sunilar to a first musical 
composition, based on objective indicia of similarity. 
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SUMMARY OF THE INVENTION 

The foregoing needs and objects, and other needs and objects that will become 
apparent from the following description, are achieved by the present invention, which 
comprises, in one aspect, a method and apparatus for classification of information and using 

5 the classification information in search and retrieval of information. In one embodiment, 
musical compositions having various highly subjective characteristics and features are 
classified according to a schema of objective attributes that are associated with the 
characteristics and features. Categorizers carry out the classification and create and store 
attribute information in a database. A search and retrieval system can receive information that 

10 characterizes the features of a desired composition and can locate a matching composition. 
As a result, information having subjective characteristics, such as musical compositions, may 
be compared and similar musical compositions can be located. 

According to one aspect, the invention provides a computer-readable medium 
comprising stored information that describes a first musical composition for use in selecting 

15 the first musical composition from among a plurality of other musical compositions that are 
similar to the first musical composition, wherein the stored information comprises a pluraKty 
of classification values that distinguish among features of the musical compositions. 

In another aspect, the invention provides a method of creating stored information that 
describes a first musical composition for use in selecting the first musical composition from 

20 among a plurality of other musical compositions that are similar to the first musical 

composition, according to a plurality of classification values that distinguish among features 
of similar kinds of musical compositions. In one embodiment, the method involves receiving 
musical information from the first musical composition and determining its features. A new 
set of the classification values are created and stored for the first musical composition based 

25 on the features that are determined. The new set of classification values are reviewed based 
on quality control criteria. 
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In another aspect, a method is provided for matching information that describes a first 
musical composition to stored information that describes a plurality of other musical 
compositions that are similar to the first musical composition, wherein the stored information 
comprises a plurality of classification values that distinguish among features of similar kinds 

5 of musical compositions. First musical information that describes the first musical 

composition and that mcludes a first set of classification values based on features of the first 
musical composition is received Second musical information that describes the plurality of 
other musical compositions is received and their features are determined. A second set of the 
classification values are created and stored for the plurality of other musical compositions 

10 based on the features that are determined. The first musical information is matched to the 

second musical information based on the classification values. A list of one or more matching 
musical compositions is created, wherein the compositions are selected firom among the 
plurality of other musical compositions based on the matching step. 

Li another aspect, the invention provides a graphical user interface useful for selecting 

1 5 a first musical composition firom among a plurality of other musical compositions that are 
similar to the first musical composition^ based on stored information that comprises a 
plurality of classification values that distinguish among features of the musical compositions, 
the graphical user interface comprising a pluraUty of graphical interface widgets that may be 
used to select a particular value for each of the classification values. 

20 In still another aspect, a method is provided of matching information that describes a 

first object to stored information that describes a plurahty of other objects that are similar to 
the first object, wherein the stored information comprises a plurality of classification values 
that distinguish among features of similar kinds of objects. First information that describes 
the first object and that includes a first set of classification values based on features of the 

25 first object is received. Second information that describes the plurality of other objects and 
determining their features is received. A second set of the classification values are created 
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and stored for the plurality of other objects based on the features that are determined. The 
&st information is matched to the second information based on the classification values. A 
list of one or more matching objects is created and stored. The matching objects are selected 
from among the plxirality of other objects based on the matching step. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example, and not by way of limitation, 
in the figures of the accompanying drawings and in which like reference numerals refer to 
similar elements and in which: 
5 FIG, 1 A is a block diagram of categorization information associated with a musical 

composition. 

FIG. IB is a block diagram of sections of categorization information associated with a 
musical composition. 

FIG. IC is a block diagram of song attribute information associated with a musical 
10 composition. 

FIG. ID is a block diagram of voice attribute information associated with a musical 
composition. 

FIG. IE is a block diagram of an alternate embodiment of classification values that 
may be associated with a musical composition. 
1 5 FIG. 2A is a block diagram of an information categorization system. 

FIG. 2B is a block diagrmi of an information search and retrieval system. 

FIG. 3 A is a flow diagram of a method of classifying information. 

FIG. 3B is a flow diagram of a method of retrieving information. 

FIG. 4A is a diagram of an exemplary graphical user interface that may be used as a 
20 form to enter song attributes and voice attributes of a musical composition. 

FIG. 4B is a diagram of an exemplary graphical user interface that may be used as a 
form to edit classification attribute values. 

FIG. 5 is a block diagram of a computer system with which an embodiment may be 

used. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

A method and apparatus for classification of information and for using the 
classifications in search and retrieval of information described. In the following description, 
for the purposes of explanation, numerous specific details are set forth in order to provide a 
5 thorough understanding of the present invention. It will be apparent, however, to one skilled 
in the art that the present invention may be practiced without these specific details. In other 
instances, well-known structures and devices are shown in block diagram form in order to 
avoid unnecessarily obscuring the present invention. 

10 CLASSIFYING INFORMATION 

One aspect of the invention relates to classifying information. In this context, 
"classifying" means creating and storing one or more information values that describe 
attributes of the information. The information values include values that describe subjective 
attributes of the information in an objective mamier in order to facilitate comparison of two 

1 5 different pieces of information and in order to facilitate identifying pieces of information that 
are similar. 

In one embodiment, the information comprises one or more musical compositions. In 
this context, "musical composition" or "song" means any audible information, including 
pre-recorded songs, albums, spoken word material, etc. However, the invention is not limited 

20 to this embodiment, and is equally applicable to other information or objects that are 

normally difficult to classify or describe in objective terms. Examples of other information or 
objects to which the invention may be applied include motion pictures, television programs, 
books, beverage, wine, works of art, perfimie, game, food, piece of apparel, or even people, 
such as those who are fashion models, photographers* models, etc. 

25 FIG. 1 A is a block diagram of categorization information associated with a musical 

composition. Musical composition 100 may be classified according to song attributes 102 
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and voice attributes 104. Song attributes 102 describe characteristics that pertain to the whole 
of a musical composition. Voice attributes 104 describe characteristics that pertain to 
individual voices that make up the musical composition. In this context, a "voice" is a 
human voice, an individual musical instrument, or another discrete sound generating device 

5 that is heard in a musical composition. Song attributes 102 and voice attributes 104 are 

created and stored in the form of one or more digital data values that are created and stored in 
anon-volatile computer memory device. 

FIG. IB is a block diagram of sections of categorization information associated with a 
musical composition. A musical composition 100 may have a plurality of sections, each of 

10 which has different song attributes. For example, a particular song may have a first section 
that is quiet and slow, followed by a second section that is loud, angry, and rapid. 
Accordingly, song attributes 102 comprise a plurality of sections 140A, 1406, 140N, etc. 
There may be any number of sections in a musical composition. Each section 140A, MOB, 
HON is associated with a stored set of song attributes that pertain to the associated section. 

1 5 FIG. 1 C is a block diagram of song attribute information associated with a particular 

section 140A of a musical composition. Song attributes 102 for section 140A may comprise, 
for example, a Tempo value 152, Weight value 154, Intensity value 156, Density value 158, 
Melodic Movement value 160, Mood Description value 162, Consonance value 164, Time 
Signature value 166, Activity value 168, Rhythm Type value 170, Rhythm Description value 

20 172, and Voices value 174. 

The foregoing values are examples of values that are suitable for objectively 
classifying subjective attributes of a song or musical composition, however, embodiments of 
the invention do not require use of exactly all such values. More or fewer values may be used 
with equal success. 

25 Tempo value 152 indicates the rate of speed of the musical composition. Activity 

value 168 indicates the amount of activity that is involved in the rhythm. Time Signature 
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value 166 indicates a time signature that are commonly used in recorded music, e.g., 4/4, 2/4, 
9/8, etc. Density value 158 indicates the number of instruments present, and thus the fulhiess 
of the sound. Weight value 154 indicates the heaviness of the composition. Consonance value 
164 indicates the resolution of the chords present in the composition. Melodic Movement 

5 value 160 indicates the degree of movement of the main voice along the musical scale. Mood 
Description value 162 indicates the overall mood of the song, such as Dreamy, Soothing, 
Fun, Depressing, Angry, Lonely, Creepy, Groovy, Uplifting, Sexy, etc. 

Rhythm Type value 170, in one embodiment, may have a value of Straight 4, Shuffle, 
Swing, Disco, Reggae, or Hip Hop Shuffle. Rhythm Description value 172, in an 

10 embodiment, describes the nature of the rhythm of the song, and may have a value of 
Rockin*, Frenetic, Steady, Throbbing, Free, Funky, Groovy, Syncopated, or Stiff. Mood 
Description value 162, in an embodiment, is selected from among Dreamy, Soothing, Fun, 
Depressing, Angry, Lonely, Creepy, Groovy, Uplifting, or Sexy. 

Voice value 174 comprises a list of one or more voices that are present in the musical 

15 composition. A set of voice attributes 104 is associated with or linked to each voice that is 
named in the Voices value 174. FIG. ID is a block diagram of voice attribute information 
associated with a musical composition. Voice attributes 104 may comprise, for example, a 
Voice Type value 106, Solo value 108, Prominence value 110, Range value 1 12, Intensity 
value 114, Ornament value 116, Flow value 118, Cleanliness value 120, Rhythmic Activity 

20 value 122, Melodic Movement value 124, Description value 126, or Harmonizes With value 
128. The foregoing values are examples of values that are suitable for objectively classifying 
voices in a musical composition, however, embodiments of the invention do not require use 
of all such values. More or fewer values may be used with equal success. 

Voice Type value 106 identifies a musical instrument, sound-making device, or a 

25 kind of a human voice such as Male Lead, Soprano, etc., associated with a voice. Solo value 
108, when set, indicates that the associated voice performs solo in the composition. 

-11- 
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Prominence value 1 10 indicates quantijfies whether the associated voice is heard in the 
acoustic foreground or background. Range value 1 12 indicates the breadth of the overall 
frequency range expressed by the associated voice. Intensity value 1 14 indicates the strength 
with which the voice is expressed. Ornament value 116 indicates whether the performance of 

5 the instrument is highly. Flow value 1 1 8 indicates whether the notes of the voice flow 
smoothly into one another. Cleanliness value 120 indicates whether the voice is pure or 
distorted. Rhythmic Activity value 122 indicates whether the rhythm of the song changes 
rapidly or slowly. Melodic Movement value 124 value indicates the degree of movement of 
the main voice along the musical scale. 

10 Description value 126, in one embodiment, is selected from among Raspy, Sweet, 

Sultry, Whiny, Forceful, Aggressive, Monotonous, Clear, Processed, Jangly, Shimmering, 
Thick, Thin, or Lush. 

In one embodiment, voice attributes 104 comprise only values for Range, Intensity, 
Cleanliness, Rhythmic Activity, or Melodic Movement. In an alternate embodiment, voice 

15 attributes 104 additionally comprise values for Prominence, Ornament, or Flow. In another 
alternate embodiment, voice attributes 104 additionally comprise the Haraionizes With value 
128, which identifies a second voice among voices in the Ust of voices with which a first 
voice harmonizes. 

In one embodiment, the song attributes comprise at least one song attribute value 
20 indicating Weight, Intensity, Melodic Movement, Range, Harmony, Density, Cleanliness, 
Flow, Melodic Movement, Mood Description, Consonance, Mood, and Dynamics. In an 
alternate embodiment, the song attributes comprise the foregoing attribute values, and also at 
least one song attribute value indicating Activity, Rhythm, Time Signature, Rhythm Type, or 
Rhythm Description, 

25 Alternatively, rather than using characterizations of Voices or instruments, a song 

may be characterized in terms of a plurality of Vocal values, each associated with one of a 
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plurality of Description values 128. Each Vocal value identifies a vocal element that is 
present in the current musical composition, selected from among a pull-down menu of 
available kinds of vocal elements. Examples of kinds of vocal elements include Male Lead, 
Female Lead, Male Backup, Female Backup, Chorus, etc. Each Description value identifies 

5 the quality of the associated vocal element. Example Description values include Raspy, 
Sweet, Sultry, Whiny, Forceful, Aggressive, Monotonous, Clear, Processed, Jangly, 
Shimmering, Thick, Thin, Lush, etc. 

- FIG. IE is a block diagram of an alternate and preferred embodunent of classification 
values that may be associated with a musical compositions, 

1 0 A musical composition 100 is associated with Rhythm values 176, Zing values 178, 

Mood values 180, and Twist values 182. Rhythm values 176 comprise Tempo value 152, 
Activity value 168, Time Signature value 166, Rhythm Description value 172, and Rhythm 
Type value 170. Zing values 178 comprise Weight value 154, Density value 158, 
Consonance value 164, and Melodic Movement value 160. Mood values 180 comprise 

15 Emotion value 182, Mood value 184, and Mood Description value 162. Twist values 182 
comprise Vocals value 184, Flavor value 186, and Language value 188. Vocals value 184 
comprises a plurality of names and descriptions of vocal elements of the composition. The 
Emotion value 182 indicates the amount of emotion expressed in the song. The Mood value 
184 indicates the strength or power with which such emotion is expressed in the song. 

20 FIG. 2 A is a block diagram of an information categorization system. In the context of 

musical compositions, the system of FIG. 2 A typically is used in an offline or preparatory 
processing stage in which one or more categorizers listen to musical compositions, determine 
features of the musical compositions, and create and store song attribute values and voice 
attribute values based on the features of the musical compositions. 

25 In this context, categorizer 200 has access to a music database 202. A categorizer 200 

may be an individual person, or an automated process, that can listen to a musical 
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composition that is stored in database 202, identify one or more features of the composition, 
and create and store attribute values based on the features. In the preferred embodiment, 
database 202 is a stored collection of digitized musical compositions or songs. Alternatively, 
database 202 is an audio reproduction device such as a compact disk player, tape deck, 
5 phonograph, etc. The precise characteristics of database 202 are not criticah What is 
important is that categorizer 200 has access to one or more musical composition for 
categorization. 

. Categorizer 200 is associated with a chent 204 having a browser 206 that is coupled 
through network 220 to classification server 208. Ghent 204 is any computer end station 

10 device such as a workstation, personal computer, network computer, etc. Browser 206 is a 
computer program executed by client 204 that can retrieve and display pages of infomation 
from server 208, and submit attribute values and related information to server 208. In the 
preferred embodiment, client 204 is an Apple Macintosh® personal computer and browser 
206 is a World Wide Web browser such as Netscape Navigator. 

1 5 Network 220 is any communication system that can carry information from chent 204 

and browser 206 to classification server 208, for example, a local area network. Intranet, or 
the global packet-switched computer network known as the Intemet. Alternatively, network 
220 may be omitted and client 204 may be coupled directly to classification server 208, 

Classification server 208 executes a classification application 210 and is coupled to a 

20 classification database 214. Classification apphcation 210 is a computer program that 
interacts with client 204 and browser 206 to assist categorizer 200 in creating and storing 
attribute values for musical compositions of database 202, according to the methods and 
mechanisms described in this document. The resulting attribute values, and related 
information, are stored in classification database 214 for later retrieval and review, and for 

25 use in search and retrieval operations. 
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Classification database 214 is any non-volatile storage system that can store and 
permit retrieval of musical composition attribute values in an organized manner. Preferably, 
classification database 214 is a commercially available relational database system, such as 
the Oracle®, Sybase®, or SQL Server database systems. Classification database 214 is 
5 appropriately programmed with a pre-defined table schema that accepts and stores the 

attribute values described above in connection with FIG. lA, FIG. IB, FIG. IC, and FIG. ID, 
or FIG. IE, In FIG. 2 A, classification database 214 is illustrated as coupled directly to 
classification server 208, however, classification database 214 may be indirectly coupled to 
classification server 208 through a network, including network 220. 

10 Classification database 2 14 is one example of a computer-readable medium 

comprising stored information that describes a first musical composition for use in selecting 
the first musical composition fi-om among a plurality of other musical compositions that are 
similar to the first musical composition, in which the stored information comprises a plurality 
of classification values that distinguish among features of the musical compositions. Other 

1 5 forms of computer-readable media may store and carry the information that is illustrated in 
Fia 1 A, FIG. IB, FIG. IC, and FIG. ID, or FIG. IE, including tables, linked lists, stacks, 
etc. The precise format or form of classification database 214 or the associated computer- 
readable media is not critical. What is important is that the information schema of FIG. 1 A, 
FIG. IB, FIG. IC, and FIG. ID, or FIG. IE is established in computer-readable form, so that 

20 values can be stored in the schema for use in classifying a particular musical composition, 
determining its similarity to other compositions, and searching and retrieving the 
information. 

In an alternate embodiment, classification database 214 comprises several separate 
databases that contain similar information, for the purpose of isolating new data from 
25 completed data that is available online for use in search and retrieval operations. For 

example, classification database 214 may comprise a development database, a production 
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database, and a live database. New attribute information and other data records are added 
only to the development database as categorization occurs. When a categorization session is 
complete, the new information in the development database is copied to the production 
database. After testing or quality control operations are carried out on the production 

5 database, its contents are copied to the live database. The live database is used online to 
receive and respond to search mxd retrieval requests. 

Moreover, the combination of classification application 210 and classification 
database 214 may also include a database interface layer, so that application 210, the 
interface layer, and database 214 are organized as a classical three-tier database application 

10 system. 

FIG. 3 A is a flow diagram of a method of classifying information. In one 
embodiment, the method of FIG. 3 A involves creating stored infoimation that describes a 
first musical composition for use in selecting the first musical composition fi'om among a 
plurality of other musical compositions that are similar to the first musical composition, 

1 5 according to a plurality of classification values that distinguish among features of similar 
kinds of musical compositions. 

In block 302, musical information fi'om the first musical composition is received and 
features of the musical composition are determined. Block 302 may involve audibly 
providing a musical composition to a categorizer, which listens to the musical composition 

20 . and determines or identifies one or more features of the musical composition. 

In block 304, a new set of the classification values for the first musical composition 
are created and stored based on the features that are determined. Block 304 may involve, for 
example, creating and storing values for a plurality of song attributes and voice attributes that 
are based on the features. The values may be entered into a database using a cUent that is 

25 coupled over a network to a server computer. The server computer executes a categorization 
application that involves generating forms and delivering the forms to the client. A 
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categorizer fills in attribute values in fields of the forms, and returns the filled-in forms to the 
server computer. The server computer extracts the attribute values from the forms and stores 
the values in the database. 

Alternatively, the values may be entered into a database or other non- volatile storage 

5 using a client computer that executes an appropriate computer program having fields for 
entering or storing values. 

In the preferred embodiment, for each musical composition, the server computer 
generates and causes the client to present a form that accepts values of song attributes for one 
of a plurality of sections of the song. Alternatively, the form may accept values of song 

10 attributes for a song without using a plurality of sections. 

FIG. 4A is a diagram of an exemplary graphical user interface 400 that may be used 
as a form to enter song attributes and voice attributes of a musical composition. 

User interface 400 generally comprises a song information pane 402 and section pane 
404. Song information pane 402 receives values that apply to a song as a whole whereas 

15 section pane 404 receives values that apply to a section of a song. 

Song information pane 402 generally comprises an Artist field 406, Song field 408, 
Status field 410, Groover field 412, Date field 414, Flag field 416, Style field 418, Style 
Status field 420, Album field 422, Location field 424, ID field 426, and Key field 428. Artist 
field 406 receives a string value identifying the recording artist who created the song that is 

20 being classified. In the example of FIG. 4A, the artist is "U2." Song field 408 receives a 

string value identifying the name and length of the song that is being classified, for example, 
" October (2:20)." Status field 410 receives a value selected firom a pull-down menu that 
indicates the status of the associated song record. Examples of Status values include Not 
Grooved, Grooved, Checked. 

25 The term " groover" is sometimes used to refer to an individual who serves as a 

classifier of songs or as categorizer 200 and "to groove" refers to a process of classifying or 
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classifying attributes of song. Groover field 412 receives a string value that identifies an 
individual who classified the current song. Date field 414 receives a date value that identifies 
the date and time at which the groover completed the grooving process. 

Flag field 416 may comprise a plurality of check-box fields that flag characteristics of 
5 the current song. For example, flags may indicate that the song is in condition for classifying 
("groovable"), has data errors or skips, contains explicit lyrics, or contains multiple musical 
sections within the piece. 

Style field 418 receives a value selected from a pull-down menu that indicates the 
genre or style of the song. Examples of style values include Alternative Pop/Rock, Post- 
10 Punk, Funk, Motown, Pop Soul, or Vocal Soul, Style Status field 420 receives a value from a 
pull-down menu that indicates whether the value of the Style field 41 8 has been approved or 
reviewed by quality control personnel or processes. Examples of Style Status values include 
Not Yet Reviewed, Inaccurate, Accurate, 

Album field 422 receives a string value that identifies an album or collection of which 
1 5 the current song or musical composition is a part. In an embodiment, Album field 422 

receives a hyperlink which, when followed using a browser, displays a list of all songs that 
are associated with that album. Thus, a categorizer can rapidly move among related songs by 
selecting the Album hyperlink and then selecting another song from those on the album. 
Location field 424 receives a string value that identifies a file system directory or other 
20 network location of a digitized file that contains the current song or musical composition. In 
an embodiment, Location field 424 receives a hyperlink to the file. Accordingly, a 
categorizer can play back a song by selecting its hyperlink from user interface 400. 

ED field 426 receives a sequence number or other identifier that is uniquely associated 
with the current section of the current song. Preferably, a value for ID field 426 is generated 
25 automatically by the system as new song sections are entered into the system. Key field 428 
receives a sequence number or other identifier that is uniquely associated with the current 
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song. Preferably, a value for ID field 426 is generated automatically by the system as new 
songs are entered into the system. 

Section pane 404 generally comprises a section identifier 430, Rhythm pane 432, 
Zing pane 434, Mood pane 436, and Twist pane 438. Each of the Rhythm pane 432, Zing 

5 pane 434, Mood pane 436, and Twist pane 438 comprises a pane label 440, status field 442, 
and identifier field 444. Pane label 440 is a system-generated label that identifies the name of 
the pane. Status field 442 receives a value from a pull-down menu that indicates the status of 
quality control review of information in the associated pane. For example, in FIG. 4A Status 
field 442 has a value of "Accurate," which indicates that the substantive values that have 

10 been entered in Twist pane 438 have been reviewed and are accurate. Identifier field 444 

provides the name of the individual or groover who entered the values in the associated pane. 

Rhythm pane 432 comprises a Tempo field 446, Activity field 448, Time Signature 
field 450, Description field 452, and Type field 454. Tempo field 446 receives a numeric 
value indicating the rate of speed of the musical composition. Activity field 448 receives a 

15 numeric value indicating the amount of activity that is involved in the rhythm. Time 
Signature field 450 receives a value from a pull-down menu of time signatures that are 
commonly used in recorded music, e.g., 4/4, 2/4, 9/8, etc. Description field 452 receives a 
value from a pull-down menu that indicates the nature of the rhythm of the song, such as 
Rockin', Frenetic, Steady, Throbbing, Free, Funky, Groovy, Syncopated, Stiff, etc. Type field 

20 454 receives a value from a pull-down menu that indicates the type or style of rhythm, e.g., 
Straight 4, Shuffle, Swing, Disco, Reggae, or Hip Hop Shuffle. 

Zing pane 434 comprises a Density field 456, Weight field 458, Consonance field 
460, and Melodic Movement field 460. Density field 456 receives a numeric value that 
indicates the number of instruments present, and thus the fulbiess of the composition. Weight 

25 field 458 receives a numeric value that indicates the heaviness of the composition. 

Consonance field 460 receives a numeric value that indicates the resolution of the chords of 
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the composition. Melodic Movement field 460 receives a numeric value that indicates the 
degree of movement of the main voice along the musical scale. 

Mood pane 436 comprises an Emotion field 464, Mood field 466, and Mood 
Description field 468. Emotion field 464 receives a numeric value that indicates the amount 

5 of emotion expressed in the song. Mood field 466 receives a numeric value that indicates the 
strength or power with which such emotion is expressed in the song. Mood Description field 
468 receives a numeric value that indicates the overall mood of the song, such as Dreamy, 
Soothing, Fun, Depressing, Angry, Lonely, Creepy, Groovy, Uplifting, Sexy, etc. 

Twist panel 438 comprises a Vocals section 470, Flavor field 476, and Language field 

10 478. The Vocals section comprises a plurality of Vocal fields 472, each associated with one 
of a plurality of Description fields 474. Each Vocal field receives a value that identifies a 
vocal element that is present in the current musical composition, selected from among a pull- 
down menu of available kinds of vocal elements. Examples of kinds of vocal elements 
include Male Lead, Female Lead, Male Backup, Female Backup, Chorus, etc. Each 

15 Description field 474 receives a value that identifies the quality of the associated vocal 
element. Example values for Description field 474 include Raspy, Sweet, Sultry, Whiny, 
Forceful, Aggressive, Monotonous, Clear, Processed, Tangly, Shimmering, Thick, Thin, 
Lush, etc. 

Flavor field 476 receives one or more values selected from among a list of flavor 
20 values that are displayed in the field and that indicate the cultural flavor of the song. 
Examples of flavor values include Latin, French, German, Asian, Indian, Irish, African, 
Caribbean, Middle Eastern, Native American, Western, Euro, etc. Language field 478 
receives a value that indicates the language in which vocals (if any) in the song are simg. 
Referring again to FIG. 3 A, in block 306, the new set of classification values is 
25 reviewed based on quahty control criteria. Block 306 may involve communicating the 

classification values from the database to a second categorizer other than the first categorizer 
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that originally created and stored the classification values. The second categorizer reviews the 
classification values while concurrently listening to the musical composition that is the 
subject of the classification values. Preferably, the second categorizer is more experienced 
than the first categorizer. This procedure tends to improve uniformity and consistency of 
5 classification values as apphed to different musical compositions. Moreover, the first 

categorizer receives rapid feedback about the accuracy of the values that have been selected 
by the first categorizer, thereby tending to train the first categorizer in a uniform manner. 

SEARCH AND RETRIEVAL 
10 Another aspect of the invention relates to a search and retrieval system usefiil in 

locating information that has been classified and determining whether a particular set of 
information having subjective characteristics is objectively similar to another set of 
information. 

FIG, 2B is a block diagram of an information search and retrieval system. In the 
15 context of musical compositions, the system of FIG. 2B typically is used in an online or 
production processing stage in which one or more clients request and retrieve infomiation 
about musical compositions, based on features of the musical compositions and song attribute 
values or voice attribute values that have been previously created and stored by a categorizer. 
In this context, digital music database 202 and classification database 214 are coupled 
20 to classification server 208, which executes a service application 212, A cUent 230 that 
executes a browser 232 is coupled through network 220 to classification server 208. One 
client 230 is shown as an example, however, any number of cUents may connect to server 
208 from any number of geographically distributed points. 

Digital music database 202 has the same structure and format as described above in 
25 coimection with FIG. 2 A. Its use in the system of FIG. 2B is optional, and it is necessary only 
when the system plays back one or more musical compositions to the client in response to a 
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search request. The system of FIG. 2B is equally useful for providing categorization 
information to the client in response to a search request, without also delivering music. 

Classification database 214 has the same structure and format as described above in 
connection with FIG. 2A. When multiple databases are used for development, production, 

5 and live use, classification database 214 comprises the live database. 

Service application 212 is a computer program executed by classilScation server 208 
that receives information requests from client 230, fomiats and submits queries to 
classification database 214, processes received results into presentable data, and returns the 
presentable data to cUent 230. Service application 212 may use the particular mechanisms 

10 described in this document, and may have other features and functions. 

FIG. 3B is a flow diagram of a method of retrieving information. Generally, FIG. 3B 
provides method of matching information that describes a first musical composition to stored 
information that describes a plurality of other musical compositions that are similar to the 
first musical composition. The stored infomiation comprises a plurality of classification 

15 values that distinguish among features of similar kinds of musical compositions. Thus, the 
method may be used to find one or more musical compositions that are similar to a first 
musical composition. 

In block 3 10, first musical information that describes the first musical composition 
and that includes a first set of classification values based on features of the first musical 

20 composition is received. Block 310 may involve receiving classification values for a musical 
composition that is liked" by a user, or for which the user wishes to find a similar musical 
composition. Altematively, block 310 may involve receiving search criteria that does not 
necessarily identify a particular existing musical composition but represents characteristics of 
a musical composition that a user wishes to retrieve. 

25 In block 312, second musical information that describes the plurality of other musical 

compositions is received, and features of the other musical compositions are determined. 
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Block 312 may involve accessing and listening to a musical composition, and detennining its 
features, in the manner described above. In block 3 14, a second set of the classification 
values for the plurality of other musical compositions are created and stored, based on the 
features that are determined. Block 314 may involve defining and storing classification 

5 values in the manner described above. 

In block 316, the fu-st musical information is matched to the second musical 
information based on the classification values. 

In one embodiment, block 316 involves carrying out a weighted matching. Each of 
the classification values may be weighted using a pre-defined weight value prior to carrying 

1 0 out the matching step. Alternatively, each of the classification values may be weighted based 
on prior matches of the first musical composition to other musical compositions that are 
associated with a particular client prior to carrying out the matching step. In another 
embodiment, the matching step comprises computing a difference value of each a pair of 
associated classification values of the first information and the second information, 

15 computing a square of the difference values, and computing a sum of the differences. 
Difference values that are within a pre-defined threshold are deemed to be the same for 
matching purposes. 

Preferably, the matching step involves matching a plurality of pairs of associated 

classification values for each musical composition. Any nimiber of pairs of classification 
20 values may be used as input to the matching process. Values may be combined in Boolean 

expressions. For example, a user can request songs that have high density with a heavy beat. 

Further, matching need not involve finding identity of all pairs of classification values. 

Matching may occur when a pre-determined subset of classification values is similar. 

Alternatively, matching may occur when a pre-determined subset of classification values is 
25 similar provided that no one pair of values is too different, according to a pre-determined 

threshold value. 
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In still another alternative embodiment, the matching step involves matching the 
musical compositions by additionally using pre-defined metadata that defines one of the 
voices in a first composition as similar to a second voice in the second composition. For 
example, the metadata may indicate that a saxophone is similar to a clarinet. When a first 

5 composition has a saxophone voice that has voice attribute values similar to the voice 
attribute values of the clarinet voice of a second composition, a match is found. 

The metadata may comprise a binary voice table that is iVby Mrows and columns in 
size, where iV= Mand represent the number of possible voices. When a cell in the table is set 
or TRUE, then instrument N is similar to instrument M. Alternatively, the table stores integer 

1 0 values that represent a degree of sinailarity or weight among two voices. A saxophone voice 
and a clarinet voice may have a relatively high similarity value, indicating close similarity, 
whereas the saxophone voice and a drum voice may have a relatively low similarity values, 
indicating difference among the voices. 

In block 318, a list of one or more matching musical compositions is created and 

15 stored. The matching musical compositions are selected firom among the plurality of other 
musical compositions based on the matching step of block 316, 

FIG. 4B is a diagram of an exemplary graphical user interface that may be used as a 
form to edit default classification attribute values. In one embodiment, the graphical user 
interface of FIG. 4B is generated by a general-pxirpose factor editor, which forms a part of 

20 classification application 2 1 0, The factor editor may be used to retrieve, review, and set 
values of various parameters that are used by the system of FIG. 2A, FIG. 2B and that are 
used as input in the graphical user interface of FIG. 4A. The editor may be used to enter 
actual factor values, or weight values. 

A factor set field 480 provides a pull-down menu that may be used to select one of a 

25 plurahty of sets of factors. A user may pull down the menu, choose a factor set name, and 
activate the selection by selecting the Select button 482. In the example of FIG. 4B, a Default 
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factor set has been selected. In response to activation of the Select button 482, classification 
application 210 displays a factor column 486 comprising a plurality of factor labels 486 A, 
486N, etc, and a value column 488 comprising a plurality of values 488A, 488N, etc. There 
may be any number of labels and associated values. 
5 The user may then change one or more of the values 488A, 488N, etc. The entered 

changes may be made persistent in the system database by selecting an Update Factor Set 
Default button 490. Thereafter, the changed values are used as default classification values in 
the graphical user interface of FIG. 4A. 

1 0 APPLICATIONS, ALTERNATIVES AND EXTENSIONS 

The list that is created and stored in block 318 of FIG. 3 A may be used for a variety 
of usefiil purposes. For example, the list may be used in an electronic commerce system to 
suggest a set of musical compositions that a prospective purchaser may like or wish to 
purchase, based on liking a first musical composition. 

15 In an embodiment, the method may involve suggesting a second musical composition 

that may be similar to the first musical composition by successively carrying out the 
matching step a plurality of times for a plurality of matching compositions, each of which is 
slightly different from the previous composition. In this way, the system can 

In another embodiment, the list is used directly to play a song that is similar to a song 

20 that the client or user knows and likes. In this embodiment, the method fiirther comprises the 
selecting and audibly playing one of the musical compositions from a stored collection of the 
musical compositions based on the list. For example, in the system of FIG. 2B, the user may 
receive a hyperlinked list of musical compositions that are accessible in digital music 
database 202. The user may select one of the compositions by selecting its hyperlink. In 

25 response, the system retrieves the selected composition and plays it using suitable audio 
circuitry and loudspeakers that are coupled to the user's computer system. 
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The foregoing method steps may be implemented in the fomi of one or more 
computer programs that form service application 212. In one embodiment, service 
application 212 is not directly accessible to clients of end users, but comprises a back end 
search and retrieval system that is accessible to clients through an application program 

5 interface (API). In this embodiment, a client application of an end user or enterprise issues 
requests to service application 212 over a network such as the Intemet in the form of function 
calls that conform to the API. Requests may be carried in the payload of an HTTP request 
that is delivered from the client over the World Wide Web to the service apphcation. Service 
application 212 parses each request and calls internal subroutines or functions as necessary to 

10 carry out the functions that are defined in the APL Service application creates and forwards a 
response message to the requesting chent. 

In still another aUemative, classification database 214 is deUvered as a stand-alone 
product to an enterprise that provides search and retrieval services. The enterprise uses its 
own front-end clients or other elements to interface to and communicate with database 214. 

1 5 Still other alternative embodiments do not require use of an intermediate network or 

the Intemet. For example, the foregoing structures and ftinctions are apphcable to an 
interactive radio application. In this application, the classification database 214 stores 
information that describes subjective characteristics about radio stations that are accessible in 
a particular geographic area. The characteristics may relate to format or other aspects of a 

20 station. Using appropriate user interface controls or software, a user can select a desired radio 
station by entering criteria such as rhythm, zing, mood, and twist attributes. 

Another alternative embodunent appUes the foregoing structures and functions to a 
personal jxike box system. In this embodiment, a user has a set of compact disks or other 
stored musical compositions, such as a collection of songs encoded usuig MPS or another 

25 digital encoding format. Classification database 214 includes information only for the songs 
and albums in the user's collection. The user executes a search and retrieval client program at 
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a desktop computer. The user enters search criteria that comprise one or more of the 
classification values. The client program searches the classification database 214 for a song 
matching criteria that are entered by the user and displays a list. The user selects one song 
from the list. The chent program retrieves the corresponding MP3 file or compact disk from 

5 the user's collection and plays it using appropriate circuitry and loudspeakers that are 
coupled to the computer. 

In an extension of this alternative, based on the search results, or in addition, the 
chent program connects the user's computer to the Internet to a system of the type shown in 
FIG. 2B. The client program then submits a search request containing the search criteria to 

10 the service appUcation 212. Search results are retrieved and displayed to the user. This 

extension also may be used in an e-commerce system. The service application 212 may be 
associated with an online merchant that sells pre-recorded musical recordings. The search 
criteria are used to identify and display musical recordings that are similar to the search 
criteria and that the user may wish to purchase. The user may then link to transactional pages 

1 5 that carry out a purchase transaction for one or more recordings in the list. 

In still another extension of the foregoing alternative, the service appUcation 212 
retrieves and delivers educational information to the user, along with a list of matching 
search results. This extension provides a complete library and education system in which 
matching information is delivered along with associated information that enhances the 

20 matching information and educates the client or user regarding the matching information. 

HARDWARE OVERVIEW 
FIG. 5 is a block diagram that illustrates a computer system 500 upon which an 
embodiment of the invention may be implemented. Computer system 500 includes a bus 502 
25 or other communication mechanism for communicating information, and a processor 504 

coupled with bus 502 for processing information. Computer system 500 also includes a main 
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memory 506, such as a random access memory (RAM) or other dynamic storage device, 
coupled to bus 502 for storing information and instructions to be executed by processor 504. 
Main memory 506 also may be used for storing temporary variables or other intermediate 
information during execution of instructions to be executed by processor 504. Computer 

5 system 500 further includes a read only memory (ROM) 508 or other static storage device 
coupled to bus 502 for storing static information and instructions for processor 504. A 
storage device 510, such as a magnetic disk or optical disk, is provided and coupled to bus 
502 for storing information and instructions. 

Computer system 500 may be coupled via bus 502 to a display 512, such as a cathode 

10 ray tube (CRT), for displaying information to a computer user. An input device 514, 

including alphanumeric and other keys, is coupled to bus 502 for communicating information 
and command selections to processor 504. Another type of user input device is cursor 
control 516, such as a mouse, a trackball, or cursor direction keys for communicating 
direction information and command selections to processor 504 and for controlling cursor 

1 5 movement on display 512. This input device typically has two degrees of freedom in two 
axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify 
positions in a plane. 

The invention is related to the use of computer system 500 for implementing the 
techniques described herein. According to one embodiment of the invention, those 

20 techniques are implemented by computer system 500 in response to processor 504 executing 
one or more sequences of one or more instructions contained in main memory 506. Such 
instructions may be read into main memory 506 from another computer-readable medium, 
such as storage device 510. Execution of the sequences of instructions contained in main 
memory 506 causes processor 504 to perfonn the process steps described herein. In 

25 alternative embodiments, hard-wired circuitry may be used in place of or in combination with 



55358-011 



-28- 



software instructions to implement the invention. Thus, embodiments of the invention are 
not limited to any specific combination of hardware circuitry and software. 

The term "computer-readable medium" as used herein refers to any medium that 
participates in providing instructions to processor 504 for execution. Such a medium may 

5 take many forms, including but not limited to, non-volatile media, volatile media, and 
transmission media, Non- volatile media includes, for example, optical or magnetic disks, 
such as storage device 510. Volatile media includes dynamic memory, such as main memory 
506, Transmission media includes coaxial cables, copper wire and fiber optics, including the 
wires that comprise bus 502, Transmission media can also take the form of acoustic or Ught 

10 waves, such as those generated during radio-wave and infi:a-red data communications. 

Common forms of computer-readable media include, for example, a floppy disk, a 
flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other 
optical medium, punchcards, papertape, any other physical medium with patterns of holes, a 
RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a 

1 5 carrier wave as described hereinafter, or any other medium from which a computer can read. 

Various forms of computer readable media-may be involved in carrying one or more 
sequences of one or more instmctions to processor 504 for execution. For example, the 
instructions may initially be carried on a magnetic disk of a remote computer. The remote 
computer can load the instructions into its dynamic memory and send the instructions over a 

20 telephone line using a modem. A modem local to computer system 500 can receive the data 
on the telephone line and use an infi-a-red transmitter to convert the data to an infira-red 
signal. An infra-red detector can receive the data carried in the infra-red signal and 
appropriate circuitry can place the data on bus 502. Bus 502 carries the data to main memory 
506, from which processor 504 retrieves and executes the instructions. The instructions 

25 received by main memory 506 may optionally be stored on storage device 510 either before 
or after execution by processor 504. 
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Computer system 500 also includes a communication interface 518 coupled to bus 
502. Communication interface 518 provides a two-way data communication coupling to a 
network link 520 that is connected to a local network 522, For example, communication 
interface 518 may be an integrated services digital network (ISDN) card or a modem to 

5 provide a data commimication connection to a corresponding type of telephone line. As 
another example, communication interface 518 may be a local area network (LAN) card to 
provide a data commimication connection to a compatible LAN. Wireless links may also be 
implemented. In any such implementation, communication interface 518 sends and receives 
electrical, electromagnetic or optical signals that carry digital data streams representing 

1 0 various types of information. 

Network link 520 typically provides data communication through one or more 
networks to other data devices. For example, network link 520 may provide a connection 
through local network 522 to a host computer 524 or to data equipment operated by an 
Internet Service Provider (ISP) 526. ISP 526 in turn provides data communication services 

1 5 through the world wide packet data communication network now commonly referred to as 
the "Internet" 528. Local network 522 and Internet 528 both use electrical, electromagnetic 
or optical signals that carry digital data streams. The signals through the various networks 
and the signals on network link 520 and through communication interface 518, which carry 
the digital data to and from computer system 500, are exemplary forms of carrier waves 

20 transporting the information. 

Computer system 500 can send messages and receive data, including program code, 
through the network(s), network link 520 and communication interface 518. In the Internet 
example, a server 530 might transmit a requested code for an application program through 
Internet 528, ISP 526, local network 522 and communication interface 518. In accordance 

25 with the invention, one such downloaded application implements the techniques described 
herein. 
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The received code may be executed by processor 504 as it is received, and/or stored 
in storage device 510, or other non-volatile storage for later execution. In this manner, 
computer system 500 may obtain application code in the form of a carrier wave. 

SCOPE 

5 Thus, methods and mechanisms for classifying information and searching and 

retrieving information have been disclosed. The methods and mechanisms may be used to 
classify distinguishing features of similar kinds of information, such as musical 
compositions. The approach is efficient and relies on a minimally necessary set of 
classification values, however, advantageously, a sufficient number of qualitative 

1 0 characteristics of the information are classified and assigned values. Lead voices of a 
composition are identified and classified, thereby improving the accuracy of similarity 
comparisons. Resulting classification data is collapsed and subjected to quality control 
measures in order to improve its consistency. The structure of the data schema helps enforce 
quality control measures. 

15 In the foregoing specification, the invention has been described with reference to 

specific embodiments thereof It will, however, be evident that various modifications and 
changes may be made thereto without departing fi:om the broader spirit and scope of the 
mvention. The specification and drawmgs are, accordingly, to be regarded in an illustrative 
rather than a restrictive sense. 

20 
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CLAIMS 

What is claimed is: 



1 1 . A computer-readable medium comprising stored information that describes a first 

2 musical composition for use in selecting the first musical composition from among a 

3 plurality of other musical compositions that are similar to the first musical 

4 composition, wherein the stored information comprises a plurahty of classification 

5 values that distinguish among features of the musical compositions. 

12. A computer-readable medium as recited in Claim 1 , wherein the classification values 
2 comprise at least one song attribute. 

13. A computer-readable medium as recited in Claim 1 , wherem the song attribute 

2 comprises at least one song attribute value indicating Weight, hitensity. Chord 

3 Movement, Range, Harmony, Density, Expressiveness, Flow, Melodic Movement, 

4 Mood Description, Consonance, Mood, Dynamics, Language, Ethnic Flavor, or Style. 

1 4. A computer-readable medium as recited in-Claim 1, wherein the song attiibute 

2 comprises at least one song attribute value indicating Flexibility, Beat, Time 

3 Signature, Rhythm Type, or Rhythm Description, or Tempo. 

1 5. A computer-readable medium as recited in Claim 1, wherein the song attribute 

2 comprises at least one rhythm type value having a value of Stiaight 4, Shuffle, Swing, 

3 Disco, Reggae, or Hip Hop Shuffle. 

1 6. A computer-readable medium as recited in Claim 1, wherein the song attribute 

2 comprises at least one rhythm description value having a value of Rockin', Frenetic, 

3 Steady, Throbbing, Free, Funky, Groovy, Syncopated, or Stiff. 
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1 7. A computer-readable medium as recited in Claim 1, wherein the classification values 

2 further comprise one or more voice values, each of which identifies one of a plurality 

3 of voices present in the first musical composition. 

1 8. A computer-readable medium as recited in Claim 1, wherein each of the voice values 

2 is at least one of Raspy, Sweet, Sultry, Whiny, Forcefiil, Aggressive, Monotonous, 

3 Clear, Processed, Jangly, Shimmering, Thick, Thin, or Lush. 

19. A computer-readable medium as recited in Clarni 1 , wherein the classification values 

2 comprise a mood value of Dreamy, Soothing, Fun, Depressing, Angry, Lonely, 

3 Creepy, Groovy, Uplifting, Sexy, Rockin' , or Neutral. 

1 10. A computer-readable medium as recited in Claun 1, wherein the classification values 

2 fiirther comprise one or more voice values, each of which identifies one of a plurality 

3 of voices present in the first musical composition, and wherein each of the voice 

4 values is associated with a plurality of voice attributes of Range, Intensity, 

5 Cleanliness, Rhythmic Activity, or MelodicMovement. 

1 11. A computer-readable medium as recited in Claim 1 , wherein the classification values 

2 further comprise one or more voice values, each of which identifies one of a plurality 

3 of voices present in the first musical composition, and wherein each of the voice 

4 values is associated with a plurality of voice attributes of Prominence, Ornamentation, 

5 Beat, Presence, or Flow. 

1 12. A computer-readable medium as recited in Claun 1 , wherein the classification values 

2 fiirther comprise one or more voice values, each of which identifies one of a plurality 

3 of voices present in the first musical composition, and wherein each of the voice 

4 values is associated with a plurality of voice attributes of Range, Intensity, 

5 Cleanliness, Rhythmic Activity, or Melodic Movement. 
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1 13. A computer-readable medium as recited in Claim 1, wherein the classification values 

2 further comprise a first voice value that identifies a first voice of a plurality of voices 

3 present in the first musical composition, and wherein the voice value is associated 

4 with a harmony value that identifies a second voice among the plurality of voices with 

5 which the first voice harmonizes. 

1 14. A method of creating stored information that describes a first musical composition for 

2 • use in selecting the first musical composition fi-om among a plurality of other musical 

3 compositions that are similar to the first musical composition, according to a plurality 

4 of classification values that distinguish among features of similar kinds of musical 

5 compositions, comprising the steps of: 

6 receiving musical information fi-om the first musical composition and determining its 

7 features; 

8 creating and storing a new set of the classification values for the first musical 

9 composition based on the features that are determined; and 

10 reviewing the new set of classification values based on quality control criteria. 

1 15. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute. 

1 16. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 

3 one song attribute value indicating Weight, Intensity, Chord Movement, Range, 

4 Harmony, Density, Expressiveness, Flow, Melodic Movement, Mood Description, 

5 Consonance, Mood, Language, Dynamics, Ethnic Flavor, or Style. 

1 17. A method as recited in Claim 1 4, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 
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3 one song attribute value indicating Flexibility, Beat, Time Signature, Rhythm Type, 

4 Tempo, or Rhythm Description. 

1 18. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 

3 one rhythm type value having a value of Straight 4, Shuffle, Swmg, Disco, Reggae, or 

4 Hip Hop Shuffle. 

1 1 9. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 

3 one rhythm description value having a value of Rockin', Frenetic, Steady, Throbbing, 

4 Free, Funky, Groovy, Syncopated, or Stiff 

1 20. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing one or more voice values, each of which 

3 identifies one of a plurality of voices present in the first musical composition. 

1 21 . A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing at least one voice values of Raspy, Sweet, 

3 Sultry, Whiny, Forcefiil, Aggressive, Monotonous, Clear, Processed, Jangly, 

4 Shimmering, Thick, Thin, or Lush. 

1 22. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing at least one mood value of Dreamy, Soothing, 

3 Fun, Depressing, Angry, Lonely, Creepy, Groovy, Uphfting, Rockin' , Neutral, or 

4 Sexy. 

1 23. A method as recited in Claim 14, wherein creating and storing the classification 

2 values comprises creating and storing one or more voice values, each of which 

3 identifies one of a plurality of voices present m the first musical composition, and 
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4 wherein each of the voice values is associated with a plurality of voice attributes of 

5 Range, Intensity, Cleanliness, Rhythmic Activity, or Melodic Movement. 

1 24. A method as recited in Claim 1 4, wherein creating and storing the classification 

2 values comprises creating and storing one or more voice values, each of which 

3 identifies one of a plurality of voices present in the first musical composition, and 

4 wherein each of the voice values is associated with a plurality of voice attributes of 

5 Prominence, Ornamentation, Beat, Presence, or Flow. 

1 25 . A method as recited in Claim 1 4, wherein creating and storing the classification 

2 values comprises creating and storing one or more voice values, each of which 

3 identifies one of a plurality of voices present in the first musical composition, and 

4 wherein each of the voice values is associated with a plurality of voice attributes of 

5 Range, Intensity, Cleanliness, Rhythmic Activity, or Melodic Movement. 

1 26. A method as recited in Claim 14, wherein creating and storing the classification 

2 values fiirther comprises creating and storing a first voice value that identifies a first 

3 voice of a plurality of voices present in the first musical composition, and wherein the 

4 voice value is associated with a harmony value that identifies a second voice among 

5 the plurality of voices with which the first voice harmonizes. 

1 27. A method of matching information that describes a first musical composition to stored 

2 information that describes a plurality of other musical compositions that are similar to 

3 the fu-st musical composition, wherein the stored information comprises a plurality of 

4 classification values tiiat distinguish among features of similar kinds of musical 

5 compositions, comprising the steps of: 

6 receiving fu-st musical information that describes the first musical composition and 

7 that includes a first set of classification values based on features of the first 

8 musical composition; 
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9 receiving second musical information that describes the plurality of other musical 

10 compositions and determining their features; 

1 1 creating and storing a second set of the classification values for the pluraUty of other 

12 musical compositions based on the features that are determined; 

1 3 matching the first musical information to the second musical information based on the 

14 classification values; and 

15 creating and storing a list of one or more matchmg musical compositions selected 

1 6 firom among the plurality of other musical compositions based on the matching 

17 step. 

1 28. A method as recited in Claim 27, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute, 

1 29. A method as recited in Claim 27, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 

3 one song attribute value indicating Weight, Intensity, Chord Movement, Range, 

4 Harmony, Density, Expressiveness, Flow, Melodic Movement, Mood Description, 

5 Consonance, Mood, Language, Ethnic Flavor, Style, or Dynamics. 

1 30. A method as recited in Claim 27, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 

3 one song attribute value indicating Flexibility, Beat, Time Signature, Rhythm Type, 

4 Tempo, or Rhythm Description. 

1 31. A method as recited in Claim 27, wherein creating and storing the classification 

2 values comprises creating and storing at least one song attribute that comprises at least 

3 one rhythm type value having a value of Straight 4, Shuffle, Swing, Disco, Reggae, or 

4 Hip Hop Shuffle. 
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32. A method as recited in Claim 27, wherein creating and storing the classification 

values comprises creating and storing at least one song attribute that comprises at least 
one rhythm description value having a value of Rockin', Frenetic, Steady, Throbbing, 
Free, Funky, Groovy, Syncopated, or Stiff, 



33. A method as recited in Claim 27, wherein creating and storing the classification 
values comprises creating and storing one or more voice values, each of which 
identifies one of a plurality of voices or instruments present in the first musical 
composition. 



34. A method as recited in Claim 27, wherein creating and storing the classification 
values comprises creating and storing at least one voice values of Raspy, Sweet, 
Sultry, Whiny, Forcefiil, Aggressive, Monotonous, Clear, Processed, Jangly, 
Shimmering, Thick, Thin, or Lush. 

35. A method as recited in Claim 27, wherein creating and storing the classification 
values comprises creating and storing at least one mood value of Dreamy, Soothing, 
Fun, Depressing, Angry, Lonely, Creepy, Groovy, UpUfting, Rockin', Neutral, or 
Sexy. 

36. A method as recited in Claim 27, wherein creating and storing the classification 
values comprises creating and storing one or more voice values, each of which 
identifies one of a plurality of voices present in the first musical composition, and 
wherein each of the voice values is associated with a plurality of voice attributes of 
Range, Intensity, Cleanliness, Rhythmic Activity, or Melodic Movement. 



37. A method as recited in Claim 27, wherein creating and storing the classification 
values comprises creating and storing one or more voice values, each of which 
identifies one of a plurality of voices present in the first musical composition, and 
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4 wherein each of the voice values is associated with a plurality of voice attributes of 

5 Prominence, Ornamentation, Beat, Presence, or Flow. 

1 38. A method as recited in Claim 27, wherein creating and storing the classification 

2 values comprises creating and storing one or more voice values, each of which 

3 identifies one of a plurality of voices present in the first musical composition, and 

4 wherein each of the voice values is associated with a plvirality of voice attributes of 

5 Range, Intensity, Cleanliness, Rhythmic Activity, or Melodic Movement. 

1 39. A method as recited in Claim 27, wherein creating and storing the classification 

2 values fiirther comprises creating and storing a first voice value that identifies a first 

3 voice of a plurality of voices present in the first musical composition, and wherein the 

4 voice value is associated with a harmony value that identifies a second voice among 

5 the plurality of voices with which the first voice harmonizes. 

1 40. A method as recited in Claim 27, fiuther comprising weighting each of the 

2 classification values using a pre-defmed weight value prior to carrying out the 

3 matching step. 

1 4 L A method as recited in Claim 27, fiirther comprising weighting each of the 

2 classification values based on prior matches of the first musical composition to other 

3 musical compositions that are associated with a particular client prior to carrying out 

4 the matching step, 

1 42. A method as recited in Claim 27, wherein the matching step comprises computing a 

2 difference value of each a pair of associated classification values of the first 

3 information and the second information, computing a square of the difference values, 

4 and computing a sum of the differences. 
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1 43 . A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among the plurality of other musical 

3 compositions based on a plurality of pairs of associated classification values. 

1 44. A method as recited in Claim 33, wherein the matching step comprises matching 

2 musical compositions selected from among the plurality of other musical 

3 compositions based on a plurality of pairs of associated classification values and by 

4 - additionally using pre-defined metadata that defmes one of the voices as similar to 

5 another of the voices. 

1 45 . A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defmes one of the rhythm types as similar to another 

5 of the rhythm types. 

1 46. A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defines one of the rhythm descriptions as similar to 

5 another of the rhythm descriptions, 

1 47. A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defines one of the rhythm time signatures as similar 

5 to another of the rhythm time signatures. 
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1 48. A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defines one of the mood descriptions as similar to 

5 another of the mood descriptions. 

1 49. A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defines one of the languages as similar to another of 

5 the languages. 

1 50. A method as recited in Claim 27, wherein the matching step comprises matching 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defines one of the ethnic flavors as similar to another 

5 of the ethnic flavors. 

1 51. A method as recited in Claim 27, wherein the matching step comprises matchmg 

2 musical compositions selected from among a plurality of other musical compositions 

3 based upon a plurality of pairs of associated classification values and by additionally 

4 using pre-defined metadata that defines one of tiie styles as similar to another of the 

5 styles. 

1 52. A method as recited in Claim 27, fiirther comprising the steps of suggesting a second 

2 musical composition that may be similar to the first musical composition by 

3 successively carrying out the matching step a plurality of times for a plurality of 

4 matching compositions, each of which is slightly different from the previous 

5 composition. 
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1 53. A method as recited in Claim 27, further comprising the steps of selecting and audibly 

2 playing one of the musical compositions from a stored collection of the musical 

3 compositions based on the list 

1 54. A graphical user interface useful for selecting a first musical composition from among 

2 a plurality of other musical compositions that are similar to the first musical 

3 composition, based on stored information that comprises a plurality of classification 

4 values that distinguish among features of the musical compositions, the graphical user 

5 interface comprising a plurality of graphical interface widgets that may be used to 

6 select a particular value for each of the classification values, 

1 55, A graphical user interface as recited in Claim 54, comprising a plurality of song 

2 attribute graphical user interface widgets, each of which is configured to set at least 

3 one song attribute value indicating Weight, Intensity, Chord Movement, Range, 

4 Harmony, Density, Expressiveness, Flow, Melodic Movement, Mood Description, 

5 Consonance, Mood, Language, Ethnic Flavor, Style, or Dynamics. 

1 56. A graphical user interface as recited in Claim 54, comprising a plurality of song 

2 attribute graphical user interface widgets, each of which is configured to set at least 

3 one song attribute value indicating Flexibility, Beat, Time Signature, Rhythm Type, 

4 Tempo, or Rhythm Description. 

1 57. A graphical user interface as recited in Claim 54, comprising a plurality of song 

2 attribute graphical user interface widgets, each of which is configured to set at least 

3 one rhythm type value having a value of Straight 4, Shuffle, Swing, Disco, Reggae, or 

4 Hip Hop Shuffle. 

1 58. A graphical user interface as recited in Claim 54, comprising a plurality of song 

2 attribute graphical user interface widgets, each of which is configured to set at least 
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3 one rhythm description value having a value of Rockin', Frenetic, Steady, Throbbing, 

4 Free, Funky, Groovy, Syncopated, or Stiff. 

1 59. A graphical user interface as recited in Claim 54, comprising a plurality of voice 

2 graphical user interface widgets, each of which is configured to set one or more voice 

3 values, each of which identifies one of a pluraUty of voices present in the first musical 

4 composition. 

1 60. A graphical user interface as recited in Claim 54, comprising a plurality of voice 

2 attribute graphical user interface widgets, each of which is configured to set one or 

3 more voice values, wherein each of the voice values is at least one of Raspy, Sweet, 

4 Sultry, Whiny, Forceful, Aggressive, Monotonous, Clear, Processed, Jangly, 

5 Shimmering, Thick, Thin, or Lush. 

1 6L A graphical user interface as recited in Claim 54, comprising a mood attribute 

2 graphical user interface widget configured to set a mood value of Dreamy, Soothmg, 

3 Fun, Depressing, Angry, Lonely, Creepy, Groovy, Uplifting, Rockin', Neutral, or 

4 Sexy. 

1 62. A graphical user interface as recited m Claim 54, comprising one or more voice 

2 attribute graphical user interface widgets configured to set one or more voice values, 

3 each of which identifies one of a plurality of voices present hi the first musical 

4 composition, and wherein each of the voice values is associated with a plurality of 

5 voice attributes of Range, Intensity, Cleanlmess, Rhythmic Activity, or Melodic 

6 Movement. 

1 63. A graphical user interface as recited in Claim 54, comprising one or more voice 

2 attribute graphical user interface widgets configured to set one or more voice values, 

3 each of which identifies one of a plurality of voices present m the first musical 
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4 composition, and wherein each of the voice values is associated with a plurahty of 

5 voice attributes of Prominence, Ornamentation, Beat, Presence, or Flow. 

1 64. A graphical user interface as recited in Claim 54, comprising one or more voice 

2 attribute graphical user interface widgets configured to set one or more voice values, 

3 each of which identifies one of a plurality of voices present in the first musical 

4 composition, and wherein each of the voice values is associated with a plurality of 

5 voice attributes of Range, Intensity, Cleanliness, Rhythmic Activity, or Melodic 

6 Movement. 

1 65. A graphical user interface as recited in Claim 54, comprising one or more voice 

2 attribute graphical user interface widgets configured to set a first voice value that 

3 identifies a first voice of a plurality of voices present in the first musical composition, 

4 and wherein the voice value is associated with a harmony value that identifies a 

5 second voice among the plurality of voices with which the first voice harmonizes. 

1 66. A method of matching information that describes a furst object to stored information 

2 that describes a plurahty of other objects that are similar to the first object, wherein 

3 the stored information comprises a plurality of classification values that distinguish 

4 among features of similar kinds of objects, comprising the steps of: 

5 receivmg first information that describes the first object and that includes a first set of 

6 classification values based on features of the first object; 

7 receiving second information that describes the plurality of other objects and 

8 determining their features; 

9 creating and storing a second set of the classification values for the plurality of other 

10 objects based on the features that are determined; 

1 1 matching the first information to the second information based on the classification 

12 values; and 

13 creating and storing a list of one or more matching objects selected firom among the 

14 plurality of other objects based on the matching step. 
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1 67. A method as recited in Claim 66, in which the object is a pre-recorded motion picture. 

1 68. A method as recited in Claim 66, in which the object is a book.. 

1 69. A method as recited in Claim 66, in which the object is a television program. 

1 70. A method as recited in Claim 66, in which the object is a beverage. 

1 71. A method as recited in Claim 66, in which the object is a work of art. 

1 72. A method as recited in Claim 66, in which the object is a perfume. 

1 73. A method as recited in Claim 66, in which the object is a human model. 

1 74. A method as recited in Claim 66, in which the object is a game. 

1 75 . A method as recited in Claim 66, in which the object is a food. 

1 76. A method as recited in Claim 66, in which the object is a piece of apparel. 
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ABSTRACT OF THE DISCLOSURE 

A method and apparatus are provided for classification of information and using the 
classification information in search and retrieval of information. In one embodiment, musical 
compositions having various highly subjective characteristics and features are classified 

5 according to a schema of objective attributes that are associated with the characteristics and 
features. Categorizers carry out the classification and create and store attribute information in 
a database. A search and retrieval system can receive information that characterizes the 
features of a desired composition and can locate a matching composition. As a result, 
information having subjective characteristics, such as musical compositions, may be 

1 0 compared and similar musical compositions can be located. 



55358-011 



-46- 



FIG.1A 



M 
MUSICAL 
COMPOSITION 

1 — c 



m 

SONG ATTRIBUTES 



m 

VOICE ATTRIBUTES 



FIG.1B 




102 

SONG ATTRIBUTES 



j4QA 
SECTION 



SECTIONS 



SECTION N 



FIG.1C 



m 

MUSICAL 
COMPOSITION 



152 
TEMPO 



WEIGHT 



INTENSITY 



DENSITY 



m 

MELODIC 
MOVEMENT 



162 
MOOD 
DE SCRIPTION 

m 

CONSONANCE 



SONG ATTRIBUTES 



I 



SECTION 



m 

TIME SIGNATURE 



168 
ACTIVITY 



170 

RHYTHM TYPE 



1Z2 
RHYTHM 
DESCRIPTION 




m 

VOICE ATTRIBUTES 



FIG.1D 



MUSICAL 
COMPOSITION 



m 

HARMONIZES WITH 



VOICE AHRIBUTES 



m 

VOICE TYPE 




m 

PROMINENCE 



112 
RANGE 



IM 
INTENSITY 



m 

ORNAMENT 



m 

FLOW 



12Q 
CLEANLINESS 



122 
RHYTHMIC 
ACTIVITY 



m 

DESCRIPTION 







12a 

Aggressive 
Clear 
Deep 
Distorted 



m. 

MELODIC 
MOVEMENT 



FIG. IE 




LANGUAGE 



FIG. 2A 



M 
CATEGORIZER 




202 
DIGITAL MUSIC 
DATABASE 





m 


CLIENT 




m 






BROWSER 





( 22Q ^ 
V NETWORK J 



m 




CLASSIFICATION SERVER 




2111 






CLASSIFICATION 






APPLICATION 





m 

CLASSIFICATION 
DATABASE 



FIG. 2B 



202 
DIGITAL MUSIC 
DATABASE 



21A 

CLASSIFICATION 
DATABASE 



m 




CLASSIFICATION SERVER 




212 






SERVICE 






APPLICATION 









NETWORK J 



m 

CLIENT 



222 
BROWSER 



FIG. 3A 





m 

RECEIVE INFORMATION 
FROM FIRST COMPOSITION 
AND DETERMINE FEATURES 










304 

CREATE AND STOFRE 
CLASSIFICATION VALUES 
BASED ON FEATURES OF 
COMPOSITION 










m 

REVIEW CLASSIFICATION 
VALUES ACCORDING TO 
QUALITY CONTROL CRITERIA 





FIG. 3B 



m 

RECEIVE FIRST MUSICAL INFORMATION THAT 
DESCRIBES A FIRST MUSICAL COMPOSITION 
AND THAT INCLUDES CLASSIFICATION VALUES 



m 

RECEIVE SECOND MUSICAL INFORMATION THAT 
DESCRIBES OTHER COMPOSITIONS, AND 
DETERMINE THEIR FEATURES 



m 

CREATE AND STORE CLASSIFICATION 
VALUES FOR THE OTHER MUSICAL 
COMPOSITIONS 



m 

MATCH FIRST MUSICAL INFORMATION 
TO SECOND MUSICAL INFORMATION 



m 

CREATE AND STORE LIST 
OF MATCHING MUSICAL 
COMPOSITIONS 




400 



J C470 "^^-/v Wif 

FIG. 4/1 



My 



All- 



Factor Editor 



Welcome, User 

Add New Factor Set 



Goto factor set: 




Now Editing Factor Set Default: 




P\G. 45 



Docket No.: 55358-012 

DECLARATION AND POWER OF ATTORNEY 



As a below named inventor, I hereby declare that: 

My residence, post office and citizenship are as stated below next to my name, 

I believe I am the original, first and sole inventor (if only one name is hsted below) or an original, first and joint 
inventor (if plural names are hsted below) of the subject matter claimed and for which a patent is sought on the 
invention CLASSIFICATION OF INFORMATION AND USE OF CLASSIFICATIONS IN SEARCHING 
AND RETRIEVAL OF INFORMATION, the specification of which 

^ is attached hereto. 

[]] was filed on as Apphcation Serial No. and was amended on (if apphcable). 

I hereby state that I have reviewed and understand the contents of the above identified specification, including the 
claims, as amended by my amendment referred to above. 

J I acknowledge the duty to disclose information which is known to me to be material to patentability in accordance 
^ with Title 37, Code of Federal Regulations, Section 1.56. 

4 1 hereby claim foreign priority benefits under Title 35, United States Code, Section 119 of any foreign 
JappUcation(s) for patent or inventor's certificate listed below and have also identified below my foreign 
J application for patent or inventor's certificate having a filing date before that of the apphcation on which priority is 
J claimed: 

^ Prior Foreign Applications(s): 

=^Number Country Day/Month/Year filed Priority Claimed 

I □ 
jl hereby claim the benefit under 35 USC § 1 19(e) of any United States provisional application(s) listed below. 
Prior Provisional Application(s): 

Application Number Filing Date 



I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States apphcation(s) 
hsted below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States apphcation in the manner provided by the first paragraph of Title 35, United States Code, Section 
112, 1 acknowledge the duty to disclose material information as defined in Title 37, Code of Federal Regulations, 
Section 1.56 which occurred between the filing date of the prior apphcation and the national or PCT international 
filing date of this apphcation: 

Prior U.S. Application(s): 

Serial No. Filing Date Status: Patented, Pending, Abandoned 

09/420,262 October 18, 1999 pending 



EXPRESS MAIL NO. aw2(^7/3 Trt;^ 



Thereby declare that all statements made herein of my own knowledge are true and that all statements made on 
information and belief are beUeved to be true; and further that these statements were made with the knowledge 
that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 
1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the vahdity of the 
appUcation or any patent issued thereon. 

I hereby appoint the following attomey(s) and/or agent(s): Edward A. Becker, Reg. No. 37,777; Stephen A. 
Becker, Reg. No. 26,527; Marcel K. Bingham, Reg. No. 42,327; John G. Bisbikis, Reg. No. 37,095; Carl L. Brandt, 
Reg. No. 44,555; Kenneth L. Cage, Reg. No. 26,151; Stephen C. Carlson, Reg. No. 39,929; Jennifer C. Chen, Reg. 
No. 42,404; Thomas A. Corrado, Reg. No. 42,439; Paul Devinsky, Reg. No. 28,553; Laura A. DonneUy, Reg. No. 
38,435; Margaret M. Duncan, Reg. No. 30,879; Brian E. Ferguson, Reg. No. 36,801; Michael F. Fogarty, Reg. No. 
36,139; Wilhelm F. Gadiano, Reg. No. 37,136; Keith E. George, Reg. No. 34,11 1; John A. Hankins, Reg. No. 
32,029; Brian D. Hickman, Reg. No. 35,894; Eric J. Kraus, Reg. No. 36,190; Edward E. Kubasiewicz, Reg. No. 
30,020; Jack Q. Lever, Reg. No. 28,149; Raphael V. Lupo, Reg. No. 28,363; Christine F. Martin, Reg. No. 39,762; 
Michael A. Messina, Reg. No. 33,424; Eugene J. MolinelH, Reg. No. 42,901; Christopher J. Palermo, Reg. No. 
42,056; Joseph H. Paquin, Jr., Reg. No. 31,647; Robert L. Price, Reg. No. 22,685; Gene Z. Rubinson, Reg. No. 
33,351; Joy Ann G. Serauskas, Reg. No. 27,952; Michele M. Schafer, Reg. No. 34,717; Arthur J. Steiner, Reg. No. 
26,106; David L. Stewart, Reg. No. 37,578; Leonid D. Thenor, Reg. No. 39,397; Keith J. Townsend, Reg. No. 
40,358; Aaron Weisstuch, Reg. No. P41,557; Edward J. Wise, Reg. No. 34,523; Alexander V. Yampolsky, Reg. 
No. 36,324; and Robert W. Zekick, Reg. No. 36,976, all of 



MCDERMOTT, WILL & EMERY 
600 13* Street, N.W. 
Washington, DC 20005-3096 

with full power of substitution and revocation, to prosecute this application and to transact all business in the 
Patent and Trademark Office connected therewith, and all future correspondence should be addressed to them. 



Full name of first inventor: Jeremy D.F. Hinman 

I Inventor's signature: ^ J F£^^^^_^ Date: I^A'c/ff 

Residence: Palo Alto, Califomia 
Citizenship: U.S. 

Post Office Address: 1 085B Tanland Drive, Palo Alto, CA 94303 



Full name of second inventor: Rolf W. Kaiser 




hiventor's signature: T^jA^ Date: /7'2D~ff 

Residence: Redwood City, 



Citizenship: U.S. 



Post Office Address: 643 Teredo Drive, Redwood City, CA 94065 



Full name of third invi 




field 



Inventor's signature: 



Date: 



Residence: Palo Alto, California 



Citizenship: U.S. 



Post Office Address: 1027 Bryant Street, #1, Palo Alto, CA 94301 



Full name of fourth inventor: Paul Deeds 



' J^—^ Date: iz./^^ /^.^ 

ey, Califomia 



Inventor's signature 



Residence: Berkeley 



Citizenship: U.S. 



Post Office Address: 2036 6th Street, Berkeley, CA 94710 



Full name of fifth inventor: Frank Fl 



Residence: Sm Franc 




a Inventor's signature: /U X ^^X^: x^/,^ /,aa^ 




Citizenship: U.S. 



IJ Post Office Address: 2566 22nd Avenue, San Francisco, CA 941 16 



Full name of sixth inventor: David giv^h 



i Inventor's signature: Date: D/gp/^^ 

Residence: ^t,^^^ Peu^^fer ^It^ucxeM. ^"^^oQ. 

Citizenship: ^ ^ 

Post Office Address: 



